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SUMMARY 


The  EEG  Systems  Laboratory  has  been  actively  improving  the  measure¬ 
ment  of  neu  r  oe  I  ec  t  r  i  c  substrates  of  human  higher  brain  funct  ions.  Out-^- 
shor  t -  term  objective  has  been  to  use  the  EEG  to  predict  decrements  in 
per  f  orman  ce  consequent  to  attentional  lapses  or  fatigue.  CkrrJ-  long¬ 
term  objective  is  to  develop  new  technologies  for  enhancing  cognitive 
ab i  I  i  t  i  es . 


Our  laboratory  continues  to  test  subjects  in  highly  controlled  para¬ 
digms.  The  paradigms  test  elementary  cognitive  and  perceptuomotor 
functions  critical  for  flying  high  performance  aircraft  and  for  per¬ 
forming  other  tasks  with  a  high  cognitive  load.  During  FY85,  record¬ 
ings  with  up  to  64  scalp  channels  have  become  routine,  as  has  the 
extended  sequence  of  signal  processing  operations  which  extracts 
minute  neurocogn i t i ve  signals  from  gigabyte  sets  of  single  trial 
data.  Algorithms  which  register  the  EEG  with  magnetic  resonance 
images  can  now  relate  the  scalp  electrode  positions  to  underlying 
cortical  structures.  Spectral  algorithms  for  filtering  eye-movement 
contaminants  have  also  been  successful  ly  developed.  Gateway  programs 
are  being  implemented  to  transport  multichannel  magne t oencepha I o - 
graphic  data  from  several  laboratories  into  our  system  for  analysis. 

Data  are  being  analyzed  from  an  interdisciplinary,  i n t e r -  I abo r a t o r y 
study  of  operational  fatigue.  This  unique  set  of  data  consists  of  57 
channe I s  of  neurophysiological,  physiological  and  behavioral  data 
recorded  from  4  Air  Force  fighter  test  pilots  performing  several  cog¬ 
nitive  and  pe r cep t uomo t o r  tasks  specially  designed  to  require  a  high 
commitment  of  attention,  memory,  judgement  and  motor  coordination. 
Recordings  we  re  made  in  3  sessions  dealing  with  task  learning,  opera¬ 
tional  fatigue  (about  16  hours  of  continuous  pe r f o r mance ) ,  and  auto¬ 
matization  of  task  performance. 

\  .. 

Initial  results  on  three  pilots  suggest  a  differential  organization 
of  preparatory  prefrontal,  premotor  and  par ieto -occipital  neuroelec- 
tric  processes  depending  upon  the  load  on  working  memory.  In  the  two 
pilots  examined  to  date,  the  ability  to  maintain  this  preparatory 
"set"  appears  to  degrade  during  fatigue. 


Robust  measurements  of  t 
between  electrodes  have 
visuomotor  judgment  task 
results  are  clear-cut  an 
models  of  the  r ap i d  I  y  s 
expectancy,  stimulus  regi 
preparation  and  execution 
response  accuracy.  Predict i 
been  identified.  These  pat  t 
had  accurate  or  inaccurate 


he  degree  of  "functional  interdependency" 
been  developed  and  app I i ed  to  a  bimanual 
recorded  from  7  right-handed  men.  The 
d  are  consistent  with  neuropsychological 
h i f t i ng  cortical  network  which  accompanies 
stration  and  feature  extraction,  response 
,  as  well  as  "updating"  to  feedback  about 
ve  cue- registered  covariance  patterns  have 
erns  distinguish  trials  which  subsequent  I y 
per  formance . 
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Along  with  pilot  analyses  of  intracerebral  recordings  conducted  in 
collaboration  with  other  laboratories  in  a  primate  model,  the  above 
results  suggest  that  it  is  possible  to  characterize  "functional 
interdependencies"  of  e ven t - r e I  a t ed  processing  between  local  neural 
areas  by  measuring  the  wave  congruence  and  lag  time  of  app r op r i a t e I y 
preprocessed  low-frequency  mac r opo t en t i a  I s .  Unique  determination  of 
the  distributed  processing  network  responsible  for  the  observed 
interdependency  patterns  is  a  formidable  problem*  This  is  the  focus 
of  our  ongoing  efforts. 
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II.  OBJECTIVES  fc  SIGNIFICANCE 
A .  General 

We  have  continued  our  development  of  new  methods  for  anal¬ 
yzing  the  electrical  activity  of  the  human  brain  during  higher  cog¬ 
nitive  functions.  Direct  measurement  of  the  mass  neural  substrate  of 
these  functions  is  one  prerequisite  to  developing  effective  means  of 
enhancing  them.  Our  aim  is  to  refine  methods  and  experimental  design 
which  will  provide  significant  steps  toward  better  understanding  of 
the  distributed  network  of  mass  neural  processes  associated  with 
human  goa I - d i r ec ted  behaviors. 

Our  short-term  objective  is  to  predict  decrements  in  performance  con¬ 
sequent  to  attentional  lapses  or  fatigue.  Our  long-term  aim  is  to  lay 
the  foundation  for  an  instrument  to  measure  human  neurocogm  t ive 
processes.  There  is  a  pressing  need  for  an  instrument  which  can  non- 
invasively  image  the  functional  neuroanatomy  of  the  human  brain  with 
high  temporal  resolution.  We  foresee  great  potential  applications  of 
this  device,  including  advanced  neuropsycho  logical  studies  of  complex 
cognitive  processes,  asses  smen  t  and  enhancemen  t  of  cognitive  skills, 
and  clinical  studies  such  as  monitoring  recovery  of  function  foil  ow¬ 
ing  head  injury.  We  be  I  ieve  that  advances  in  recording  EEG  and  MEG 
signals,  coupled  with  application  of  digital  signal  processing  tech¬ 
niques,  now  make  feasible  the  basic  research  which  will  lead  to 
development  of  an  'electromagnetic  brain  scanner." 


Recordings  with  up  to  64  scalp  EEG  channels  during  highly 
controlled  tasks  are  now  routine  in  our  laboratory,  as  is  the 
sequence  of  signal  processing  operations  r-quired  to  extract  neuro- 
cognitive  info  r ma  t  i on  f  r  om  sets  of  single-trial  data.  More  robust 
measures  of  the  "functional  interdependency"  (cross covariance  and 
delay  t  i me )  be  tween  electrodes  have  been  developed  and  applied  to 
data  from  a  bimanual  visuomotor  task  recorded  several  years  ago.  The 
results  are  clear-cut  and  are  consistent  with  neuropsychological 
models  of  the  cortical  activity  accompanying  expectancy,  stimulus 
registration  and  feature  extraction,  movement  preparation  and  exe¬ 
cution,  and  feedback  (see  Section  IV).  Many  of  these  results,  along 
with  pilot  analyses  of  intracerebral  recordings  using  a  primate 
model,  cannot  be  explained  by  single  equivalent-current  dipole  source 
models,  but  could  be  understood  in  the  context  of  multiple,  simultan¬ 
eously  active  neuronal  sources.  However ,  determination  of  the  mul¬ 
tiple  sources  of  neural  activity  associated  with  cognitive  processes 
remains  a  formidable  problem,  since  their  signals  overlap  both  in 
time  and  in  space,  and  are  usually  smaller  than  "background"  brain 
act  i v i  t  y . 

Since  the  main,  unresolved  issue  is  the  origin  of  interdependency 
patterns  measured  at  the  scalp,  we  are  now  concentrating  on  improving 
our  ability  to  relate  scalp  recordings  to  source  generator  configura¬ 
tions.  With  improved  technologies  currently  available,  we  believe 
that  several  simultaneous  sources  can  be  resolved. 


I  I  I  • NEUROCOGN I T I VE  PATTERN  (NCP)  ANALYSIS 


A ,  Overview 


The  generic  term  "NCP  Analysis*  refers  to  a  mu  I t i - s t epped 
procedure  developed  to  extract  task-related  spat i otempora I  patterns 
from  the  unrelated  “noise*  of  the  brain.  There  have  been  three  prior 
generations  of  NCP  Analysis.  The  first  (circa  1977)  measured  back¬ 
ground  EEG  spectral  intensities  while  people  performed  complex  tasks, 
such  as  reading  and  writing.  The  second  (circa  1980)  measured  single¬ 
trial,  zero-lag  crosscorrelat ions  between  91  pairwise  combinations  of 
15  electrodes  recorded  during  performance  of  spl it-second  perceptuo- 
motor  tasks.  The  third  (circa  1984)  measured  e ven t - r e I  a t ed  lagged 
c r osscovar i ances  between  ali  pairwise  combinations  of  up  to  26  elec¬ 
trodes  on  enhanced  averages  obtained  from  a  control  led  sequence  of 
stimuli  in  which  a  person  prepared  for  and  executed  perceptual  judge¬ 
ment  and  motor  control  tasks  and  received  performance  feedback 
(Gevins  et  a!.,  1984a, b;  see  below).  This  generation  is  also  being 
applied  to  i n t r ace r eb r a  I  recordings  from  primates.  A  fourth  genera¬ 
tion  is  currently  under  development.  In  this  generation,  because  of 
the  large  size  of  the  single-trial  data  sets  (up  to  150  megabytes  for 
each  person),  two  passes  through  the  data  are  required  to  complete 
the  analysis.  The  first  pass  selects  channels,  intervals  and  trials 
with  task-related  information  to  reduce  the  amount  of  data  prior  to 
measuring  “functional  interdependencies"  (similarity  and  time- 
alignment  of  waveshapes)  between  channels.  The  second  pass  measures 
interdependencies  between  channels  on  averages  obtained  from  the 
reduced  data  set.  NCP  Analysis  is  implemented  in  the  ADIEEG-IV 
analysis  system  (Fig.  1;  see  Gevins,  In  press, a,  for  discussion  of 
c r os sco r r e I  a t  i on  analysis,  and  Gevins,  In  press, b,  for  discussion  of 
statistical  pattern  recognition). 

B .  Cu  r  r  en  t  Procedures  (Fig.  2 ) 

1.  Record  up  to  64  channels  of  data  and  apply  the  Lap  I ac i an 
operator  to  the  potential  measurements  to  obtain  the  reference- 
independent  second  difference  of  potential  at  each  non-per ipheral 
scalp  electrode  location. 

2.  Create  a  single,  a r t i f ac t - f r ee  data  set  which  will  be 
compared  with  a  reference  "baseline’  data  set.  Alternatively,  create 
two  data  sets  that  differ  according  to  a  specified  criterion,  such  as 
the  hand  used  for  response,  but  which  are  otherwise  balanced  for 
stimulus,  response  and  pe r f o rmance - r e I  a t ed  factors.  In  either  case, 
the  two  data  sets  represent  conditions  which  are  to  be  compared. 
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AOEEO  IV  NEUROCOONTTIVE 
PATTERN  RECOONTTION  8Y8TEM 

RECORD  DATA 


MOOG.  SOURCES  OS  EVENT-RELATED  PATTERNS 


Fig.  1.  ADIEEG-IV  System  for  quan t i f i cat i on  of  event  -  re  I ated  brain 
signals*  Separate  subsystems  perform  on-line  exprimental  control  and 
data  collection*  data  selection  and  evaluation,  signal  processing  and 
pattern  recognition.  Current  capacity  is  64  channels,  Digital  data 
tapes  from  other  laboratories  are  converted  into  the  AO  I EEG  data 
format  using  gateway  programs;  they  are  then  processed  using  the  same 
program  modules  as  data  collected  in  the  EEG  Systems  Laboratory.  The 
system  is  currently  implemented  on  a  PDP11  and  two  Masscomp  computers 
which  are  connected  by  high-speed  buses,  The  RSX  and  UNIX  operating 
systems  are  used, 


NEUROCOGNITIVE  PATTERN  ANALYSIS 

1.  SELECT  TRIALS  WITH  EVENT-RELATED  SIGNALS 

2.  COMPUTE  ENHANCED  AVERAGE  ERPS  AND  FILTER 

3.  DEFINE  WINDOWS  FOR  COVARIANCES  FROM  ERP  PEAKS 

4.  COMPUTE  COVARIANCES  AND  DELAYS  BETWEEN  PAIRS 
OF  CHANNELS 

6.  GRAPH  MOST  SIGNIFICANT  COVARIANCES  IN  EACH 
WINDOW 
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Fig.  2(b)  .  I  I 
elect  rode  pos i t i 
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lustration  of  the  covariance  diagram.  indicating 
ons  and  showing  the  relationship  of  a  covariance  line 
filtered  averaged  even t - r e I  a t ed  timeseries  from  the  2 
lec-  trodes.  In  all  covariance  diagrams,  the  width  of 
the  magnitude  of  covariance  between  2  electrodes  and 
cates  the  magnitude  of  time  delay.  When  the  color  of 
the  same  as  the  covariance  line,  the  covariance  is 
the  arrow  is  gray,  the  covariance  is  negative.  The 
rom  leading  to  lagging  channel.  (Specific  values  are 
ag  r  am . ) 
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Signal  enhancement: 

3.  Compute  average  event  -  re  I ated  timeseries  for  stimulus* 
response*  cue  or  feedback  for  each  person*  and  use  the  latencies  of 
peaks  to  determine  centerpoints  of  analysis  windows  for  pattern 
recognition  analysis  on  the  sets  of  single  trials.  Adjust  the  window 
size  to  span  the  peak  (usually  125  to  250  msec ).  (Set  other  wi ndows 
at  stimulus  and  response  times.) 

4.  Apply  a  7  Hz  low-pass  filter  to  the  timeseries  and  then 
decimate  appropriately.  Use  3-7  timeseries  values  as  features. 

5.  Partition  the  even t - r e I  at ed  and  ‘baseline*  data  (or  the 
data  of  the  two  conditions)  into  trai.ning  and  validation  sets.  For 
each  channel*  apply  a  I ayered-networ k  pattern  recognition  algorithm 
(a  type  of  discriminant  ana  I y s i s - - - see  Qev i ns  1980*  1984*  In  press*b) 
to  generate  equations  which  represent  invariant  differences  between 
event  -  re  I ated  and  ‘baseline"  data  sets  or  between  two  conditions  in 
the  analysis  interval. 

6.  Test  the  equations  with  the  validation  data  to  determine 
which  electrodes  differ  significantly  between  conditions.  Set  the 
level  of  significant  classification  well  above  that  attained  with  the 
data  randomly  assigned  to  two  pseudo - cond i t i ons ,  as  determined  by 
separate  analysis. 

7.  Draw  diagrams  showing  which  electrodes  contain  task- 
related  information  in  each  interval. 

8.  Collect  trials  with  task-related  signals  (i.e.,  trials 
correctly  classified  in  the  validation  set)*  and  compute  ‘enhanced” 
averages  using  those  trials  which  were  correctly  classified  in  a  ma¬ 
jority  of  the  significant  channels.  Graph  isopotential  contours  at 
selected  time  points. 

Be  tween - channe I  interdependency  analysis: 

9.  Filter  enhanced  averages  using  Gaussian  FIR  filters  in 
frequency  bands  determined  from  a  Wigner  distribution  of  the  enhanced 
averages.  Compute  multi-lag  crosscovariance  functions  between  all 
channel  pairs  in  each  analysis  window.  Window  centerpoints  are  deter¬ 
mined  from  peak  latencies  and  window  widths  are  derived  from  the 
dominant  frequency  of  each  band. 

10.  Compute  a  delay  estimate  for  each  pair  from  the  lag  with 
maximum  absolute  covariance.  Compute  the  significance  of  this  covar¬ 
iance  by  comparison  to  a  reference  noise  distribution  derived  from 
the  averages  themselves  (see  below).  Display  most  significant 
channe I  -  pa i r s  and  their  time  delays. 

The  above  method  of  computing  covariances  from  enhanced  averages  was 
used  to  generate  the  new  results  in  Section  IV,  A  related  method 
using  unfiltered  correlations  between  intracerebral  electrodes  was 
applied  to  primate  data. 


1.  Complex  perceptuomotor  and  cognitive  activities  such  as 
reading  and  writing  produce  unique*  spatially  d i f f erent i ated  scalp 
EEG  spectral  patterns.  These  patterns  had  sufficient  specificity  to 
identify  the  type  of  task  (Gevins  et  al.,  1979a).  These  results  agree 
with  previous  reports  of  hemispheric  I atera I i zat i on  of  "spatial*  and 
"linguistic"  processing. 

2.  When  tasks  are  controlled  for  stimulus*  response  and 
per f ormance- re  I ated  factors*  complex  cognitive  activities  such  as 
arithmetic*  letter  substitution  and  mental  rotation  have  identical* 
spatially  diffuse  EEG  spectral  scalp  distributions.  Compared  with 
staring  at  a  dot,  such  tasks  have  approximately  10%  reductions  in 
alpha-  and  beta-band  spectral  intensities  (Gevins  et  al.*  1979b, c). 
This  reduction  may  be  an  index  of  task  workload.  Since  no  patterns  of 
hemispheric  lateralization  were  found*  this  study  suggests  that  prev¬ 
ious  and  most  current  reports  of  EEG  hemispheric  lateralization  may 
have  confounded  electrical  activity  related  to  limb  and  eye  movements 
and  arousal  with  those  of  mental  activity  per  se  (Gevins  et  al.* 
1980  )  . 

3.  Split-second  visuomotor  tasks,  controlled  so  that  only 
the  type  of  judgment  varied,  are  associated  with  complex,  rapidly 
shifting  patterns  of  i n t e r - e I ec t r ode  correlation  of  single-trial 
brain  potential  timeseries.  Differences  between  spatial  and  numeric 
judgments  are  evident  in  the  task-cued  prestimulus  interval,  Complex 
and  often  lateral i zed  patterns  of  task-related  correlation  changed 
with  split-second  rapidity  from  stimulus  onset  to  just  prior  to 
response,  at  which  time  there  was  no  difference  between  tasks  (Gevins 
et  al.,  1981).  This  suggests  that  once  t as k - spec i f i c  perceptual  and 
cognitive  processing  is  completed,  a  motor  program  common  to  both 
tasks  is  being  executed. 

4.  Rapidly  shifting  focal  scalp  patterns  of  differences 
between  tasks  can  be  extracted  with  NCP  Analysis.  The  move  and  no¬ 
move  variants  of  a  split-second  visuospatial  judgment  task,  which 
differed  in  expectation,  in  type  of  judgment,  and  in  response  (move 
or  no-move),  were  associated  with  distinctly  different  patterns  of 
single-trial  evoked  correlation  between  channels  (Gevins  et  al., 
1983,  1985).  These  results  are  consistent  with  neuropsychological 
models  of  analogous  tasks  derived  from  functional  deficits  of 
patients  with  focal  lesions.  Further,  they  suggest  that  although 
simple  perceptuomotor  tasks  are  associated  with  a  complex  dynamic 
mosaic  of  brain  potential  patterns,  it  is  possible  to  isolate  foci  of 
maximal  differences  between  tasks.  It  is  clear  that  without  a  split- 
second  temporal  resolution  it  would  not  be  possible  to  isolate 
rapidly  shifting  lateral i zat ions  and  localizations  which  are  presum¬ 
ably  associated  with  successive  stages  of  task-related  processing. 

5.  The  focal  patterns  of  evoked  correlation  derived  by  NCP 
Analysis  significantly  distinguished  the  single-trial  data  of  7  of 
the  9  people  in  the  above  study.  This  suggests  that  consistent  neuro- 
cognitive  processes  were  measured  in  the  majority  of  right-handed 
male  research  participants  (Gevins  et  al.,  1985). 
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6.  Analyses  of  pilot  intracerebral  primate  data  (recorded  at 
Rebert's  laboratory  at  SRI)  revealed  rapidly  shifting  correlation 
patterns  between  h i ppocampus (  substantia  nigra#  premotor  cortex# 
vent roanter i or  nucleus  of  the  thalamus#  and  the  midbrain  reticular 
formation  which  distinguished  go  and  no-go  visuomotor  tasks.  Inter¬ 
dependencies  of  the  hippocampus  with  other  loci  were  prominent  in  the 
"go"  condition  (which  lead  to  reward)  but  not  in  the  no-go  condition 
during  intervals  from  240  to  578  msec  after  the  cue.  The  patterns 
were  cha r ac t e r i z ed  by  delays  of  up  to  72  msec  or  more  between  loci. 


Compar  i son  o  f  NCP  Ana 
T opoar aph i c  Maps 
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Our  experiments  and  analyses  are  grounded  on  the  body  of 
information  gained  from  ERP  research  and  have  the  same  underlying 
goal:  to  resolve  spatially  and  temporally  overlapping,  task-related 
neural  processes  (Chapters  3#  5#  10#  12#  13#  14,  16#  17  8c  18  in 
Gevins  8c  Remond ,  In  press).  Our  approach  differs  in  several  ways  from 
the  currently  popular  ERP  paradigms  which:  (a)  extract  a  few  features 
from  averaged  ERPs  by  peak  picking  or  principal  components  analysis 
(PCA);  and  (b)  perform  hypothesis  testing  on  the  features  with  ANOVA. 
First,  because  neu r ocogn i t i ve  processes  are  complex,  we  are  concerned 
with  spa t  i o t empor a  I  task-related  activity  recorded  by  many  (currently 
up  to  64)  scalp  electrodes  in  many  (currently  about  20)  time  inter¬ 
vals  spanning  a  5-second  period  extending  from  before  a  cue#  through 
stimulus  and  response#  to  presentation  of  feedback.  By  contrast#  most 
ERP  experiments  are  concerned  with  measurement  of  one  to  a  few  “com¬ 
ponents"  associated  with  a  single  stimulus  or  response  registration 
in  a  few  channels  in  an  epoch  under  one  second.  Additionally#  NCP 
Analysis  quantifies  neu r ocogn i t i ve  activity  in  terms  of  a  variety  of 
parameters#  including  measurement  of  "functional  interdependencies" 
between  channels#  rather  than  just  amp litude  and  latency  of  ERP  com¬ 
ponents.  This  increased  d i mens ionality  of  par  ame  ter  i zat  ion  may  f  ac i  I  - 
itate  the  measurement  of  more  subtle  aspects  of  neurocogm  t  ive 
processes . 


Second#  the  questionable  assumption  of  a  multivariate  normal  distri¬ 
bution  of  very  brief  brain  potential  timeseries  is  not  made  in  the 
first  signal  enhancement  pass  of  NCP  Analysis#  whi le  robust  stat¬ 
istics  are  used  in  the  second  pass  to  assess  the  significance  of 
covariances.  Third#  feature  extraction  3nd  hypothesis  testing  are 
performed  as  a  single  process  which  determines  the  differences  in 
signal  properties  between  the  conditions  of  an  experiment,  or  their 
differences  with  respect  to  a  "baseline.”  We  think  th3t  this  is  more 
effective  than  feature  extraction  techniques#  such  as  PCA,  which  are 
based  on  possibly  irrelevant  criteria  (such  as  statistical  indepen¬ 
dence  of  features  regardless  of  discriminating  power).  Fourth#  in 
NCP  Analysis  task-related  patterns  of  consistency  can  be  extracted 
f  r  om  sets  of  single-trial  data.  Significant  results  may  be  obtained 
as  long  as  there  is  a  pattern  of  consistent  difference#  even  when  the 
means  of  the  two  data  sets  are  not  significantly  different. 


NCP  Analysis  should  also  be  distinguished  from  attractive  topographic 
color  displays  of  the  individual  time  points  of  16-20  channels  of 
averaged  ERPS  or  EEG  spectra#  or  the  difference  between  such  measures 
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and  a  set  of  normative  data.  NCP  Analysis  uses  extensive  signal  proc¬ 
essing  and  pattern  recognition  algorithms  on  up  to  64  channels  to 
extract  minute  "neurocogni t ive"  signals  from  the  unrelated  background 
'noise'  of  the  brain,  computes  be t ween - channe I  interdependency  pat¬ 
terns  and  displays  their  scalp  distribution  in  topographical  dia¬ 
grams.  (See  Gevins  k  Remond,  In  press,  for  a  thorough  treatment  of 
all  types  of  brain  electromagnetic  signal  analyses.) 


IV.  RECENTLY  COMPLETED  AND  ON-GOING  EXPERIMENTS 
A .  B i manua I  Nume  r  i c  V i s  uomo  tor  E  x  oe  r  i men  t 

The  aim  of  this  experiment,  the  recordings  of  which  were 
largely  funded  by  the  Office  of  Naval  Research  ( ONR ) ,  was  to  measure 
the  rapidly  changing,  spatially  distributed,  neu r ocogn i t i ve  patterns 
associated  with  execution  of  precise  right-  and  left-hand  finger 
movements  (Diewert  k  Stelmach,  1978;  Newell,  1978;  Sternberg  et  al . , 
1978;  Rosenbaum  et  al.,  1984)  during  complete  5  -  second  -  I ong  task 
trials,  from  cued  preparation,  through  pos t - s t i mu  I  us  perceptual  and 
cognitive  processing,  ard  response  execution,  to  the  'updating'  asso¬ 
ciated  with  feedback  (Fig.  3). 

EEG  signals  from  26  scalp  electrodes  referenced  to  a  midi  me  antero- 
parietal  electrode,  vertical  and  horizontal  eye  movements  ( EOG ) ,  and 
f  I e x o r  d i q i  tor  i  muscle  potentials  ( EMG )  were  digitized  on  magnetic 
tape  at  128  Hz  from  .75  sec  before  the  cue  to  one  second  after  feed¬ 
back.  The  Lap  I ac i an  operator  was  applied  to  the  EEG  potentials  by 
computing  the  curvature  of  the  potential  field  about  each  electrode. 
Two  independent  raters  edited  the  data  for  artifacts  by  visual 
inspection  of  EEG,  EOG  and  EMG  polygraph  channels.  Trials  with  any 
sign  of  artifact  we  re  eliminated,  as  we  re  trials  with  si ow ,  b i mo da  I 
or  delayed  responses,  and  trials  with  any  sign  of  f I exo r  diqi  tor  i 
activity  in  the  cue- to-st imu I  us  epoch.  For  each  event,  comparison 
bet  ween  conditions  (e.g.»  r ight-vs- left  hand  mo  v erne n  t  )  ,  was  per  for  med 
on  artifact-free  trials,  balanced  between  conditions  for  relevant 
behavioral  variables  (e.g .,  response  onset  time,  response  movement 
force,  velocity,  acceleration  and  duration). 

Spa t  i o t empo r a  I  neuroelectric  patterns  were  quantified  by  measuring 
the  covariance  (similarity  of  waveshape  and  timing)  over  brief  inter¬ 
vals  of  highly  processed  averaged  even t - r e I  a t ed  timeseries  recorded 
f  r  om  different  locations.  Be  tween -channel  covariances  fr  om  f  i  I  t  e  r  ed 
averages  were  examined  in  two  frequency  bands,  "delta"  (0.1-3  Hz)  and 
“theta"  (4-7  Hz),  during  25  intervals  from  the  cue  to  after  the  feed¬ 
back.  Delays,  defined  as  the  lag  time  of  maximum  covariance,  were 
examined  to  plus  or  minus  125  msec.  Unless  otherwise  noted,  al I 
covariances  described  we  re  f  r  om  "theta-band"  filtered  averages.  Since 
this  was  the  first  complete  set  of  data  available  for  our  new  anal¬ 
yses,  the  specific  results  are  briefly  described  here  as  illustrative 
of  the  me  thods.  Unless  otherwise  stated,  all  covariances  we  re  f  r  om 
theta-filtered  waveforms. 

1  .  Prestimulus  Preparatory  Set  .  Neurocogni  tive  patterns 
related  to  differential  preparation  of  right-  and  left-hand  responses 
were  measured  in  the  cue- to-st imul us  epoch.  The  7-person  cue- 
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registered  Lap  I ac i an  t imeser ies  contained  two  distinct  waves  corre¬ 
sponding  to  the  early  and  late  components  of  the  Contingent  Negative 
Variation  ( CNV ) s  a  broad  “0  wave'  between  250  and  500  msec  followed 
by  a  slow  ramp-like  shift  leading  to  the  stimulus.  Evidence  of  a  pre¬ 
stimulus  hand  -  spec i  f i c  set  occured  in  covariance  patterns  for  "delta- 
band"  (0.1-3  Hz)  filtered  data  in  the  500  msec  preceding  the  stimulus 
during  the  late  CNV  wave.  The  most  obvious  difference  between  condi¬ 
tions  was  the  predominantly  left-sided  patterns  for  right  cue.  com¬ 
pared  to  the  mixed  patterns  for  left  cues.  These  patterns  tended  to 
focus  on  particular  frontal.  central  and  parietal  electrodes  over 
areas  consistent  with  cortical  loci  of  preparatory  motor  sets  sug¬ 
gested  by  clinical  neuropsychological  studies  (Fig.  4).  Different 
patterns  were  found  for  trials  which  had  subsequently  accurate  or 
inaccurate  visuomotor  performance  (Fig.  9).  suggesting  that  a  prep¬ 
aratory  set,  composed  of  an  attentional  and  a  somes t he t i c -mo t o r  com¬ 
ponent,  is  necessary  for  good  performance.  To  further  verify  the 
cerebral  origin  of  the  covariance  patterns,  EOG  and  EMG  channels  were 
submitted  to  a  mathematical  pattern  classification  program.  The 
objective  was  to  determine  if  these  channels  could  significantly 
classify  right-hand  accurate  from  right  inaccurate  trials,  and 
left-hand  accurate  from  left  inaccurate  trials,  The  eye  movement 
channe I s  we  re  delta  filtered,  and  down  samp  led  to  16  samp les/sec  over 
an  interval  375  msec  wide  with  a  687  msec  center;  EMG  channels  were 
half-wave  rectified.  Neither  the  EOG  nor  EMG  channels  significantly 
classified  trials  at  the  level  better  than  chance  (SOX). 

2  •  Pos  t  -  s  t  i mu  I  us  Process i nq  o  f  Move  and  No  -  mo ve  Trials. 
Differences  between  move  and  no-move  trials  were  examined  in  post¬ 
stimulus  intervals  spanning  early  and  late  event-related  waves. 
Right-  and  left-cued  no-move  trials  were  examined  for  differences 
related  to  miscued  hand  (violation  of  preparatory  set).  Covariance 
patterns  in  early  intervals  were  similar  for  right-  and  left-hand 
move  trials  (Fig.  5),  until  the  interval  centered  at  312  msec,  where 
they  became  highly  complex.  Covar iance  patterns  for  right-  and  left- 
cued  no -move  trials  differed  from  move  patterns  from  the  first  post¬ 
stimulus  interval,  but  they  were  quite  similar  to  each  other  by  the 
interval  centered  at  312  msec.  During  this  interval  which  spanned  the 
large  early  P300  wave  and  inc'uded  a  portion  of  the  latter  P300  wave, 
lateral  frontal  and  mi d I i ne  parietal  electrodes  lead  aCz,  wh ich  in 
turn  I ead  F  z  (Fig,  6 )  . 

3.  Be  tween -Hand  Movemen  t  Analysis.  Differences  between  right 
and  left-hand  move  trials  were  examined  in  response-registered  data 
in  intervals  before  and  after  response  onset.  The  7-person  waveform 
was  largest  at  aCz  and  consisted  of:  (1)  a  slow  r  amp -like  shift 
beginning  about  450  msec  before  movemen  t  onset  (Readiness  Potental), 
(2)  an  increase  in  slope  at  EMG  onset  (Motor  Potential)  which  peaked 
62  msec  after  response  onset,  and  (3)  the  Response  After -Potent i al 
187  msec  after  response  onset,  This  anterior  mid  line  precentral  elec¬ 
trode,  overlying  the  pre- mo  tor  supplementary  motor  cortices,  was  the 
focus  of  all  movement-related  covariance  patterns,  and  there  was  a 
consistent  long-delay  covariance  bet  we  en  the  anterior  mid  line,  pre¬ 
central  and  midline  prefrontal  electrodes  in  these  patterns  (Fig.  7  I  . 
The  pattern  for  the  Motor  Potent 'al  clearly  reflected  the  sharply 
focused  current  sources  and  sinks  spanning  the  hand  areas  of  motor 
cortex.  Both  the  Lap  I ac i an  topography  and  the  covariance  patterns 
suggest  distinct  source  generator  configurations  for  the  Readiness 
Potential,  Motor  Potential  and  Response  After-Potential. 
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4.  Feedback  Ana Ivsisi  Neu r ocogn i t i ve  patterns  following 
"win"  and  'lose'  feedback  for  move  trials  were  examined  for  differ¬ 
ences  in  “amount  of  updating”  and  hand  -  spec i f i c  effects.  These 
results  were  compared  to  feedback  for  no-move  trials,  where  no  "motor 
system  updating'  was  required.  Evidence  was  found  in  feedback- 
registered  covariance  patterns  and  averaged  Laplacian  timeseries 
which  may  reflect  different  degrees  of  updating  of  motor  systems  when 
given  information  about  response  accuracy.  Feedback  trials  in  which 
no  ad  j us t men  t  of  the  mo  tor  system  was  required  elicited  very  sma I  I 
P 3 0 0  peaks.  Feedback  for  accurate  responses  elicited  a  larger  ampli¬ 
tude  "early  P300."  while  feedback  for  inaccurate  responses  elicited  a 
larger  amplitude  in  the  later  P30Q.  The  covariance  pattern  in  the 
interval  spanning  these  waves  was  far  more  complex  for  inaccurate 
trials.  involving  foci  of  leading  covariances  at  parietal  sites,  of 
lagging  covariances  over  dorsolateral  prefrontal  areas,  and  mixed  de¬ 
lays  at  the  anterior  precentral  site  (Fig.  8).  It  would  be  difficult 
to  explain  these  different  patterns,  which  included  delays  of  up  to 
80  msec  between  electrodes  3  cm  apart,  solely  in  terms  of  volume- 
conducted  activity  from  one  or  two  subcortical  and/or  cortical 
generators. 

B .  Fat  i cue  E xpe r  i men  t 

1.  Qve  r  v i ew .  Data  are  currently  being  analyzed  from  an 
i  n t e r d  i  sc i p I  i na r y ,  i n t e r  -  I  abo r a t o r y  study  of  operational  fatigue. 
This  unique  set  of  data  consists  of  57  channels  of  neurophys  10I  - 
ogical ,  ph y s i o I og i ca I  and  behavioral  data  recorded  from  several  Air 
Force  fighter  test  pilots  performing  cognitive  and  perceptuomotor 
tasks  specially  designed  to  require  a  high  comm itment  of  attention, 
memory,  judgement  and  motor  coo r d i na t i on .  Recordings  were  made  in 
four  sessions,  on  three  days,  dealing  with  task  learning,  operational 
fatigue  (about  16  hours  of  continuous  performance),  and  automati¬ 
zation  of  task  performance. 

Initial  results  on  three  pilots  suggest  a  differential  organization 
of  preparatory  prefrontal,  premotor  and  par ieto-occipi tal  neuroelec- 
tric  processes  depending  upon  the  load  on  working  memory.  In  the 
first  two  pilots  ex  am ined  so  far,  the  ability  to  ma intain  this  prep¬ 
aratory  "set"  appears  to  degrade  during  fatigue. 


The  job  performance  of  a  fatigued  person  may  be  described  as  'having 
no  film  in  the  came  r  a .  "  It  i s  a  common  observation  that  persons  in 
such  a  condition  may  be  able  to  acceptably  perform  rote  behaviors, 
but  are  unable  to  respond  to  sudden  unexpected  situations  which 
require  full  comm i  t  men  t  of  higher  cognitive  functions.  Further  mo  r  e , 
fatigue  can  produce  momentary  lapses  of  attention,  decision  processes 
and  perceptuomotor  performance.  One  characteristic  of  fatigue  is  a 
reduced  ability  to  perceive  ones  diminished  capacity.  Since  there 
are  many  situations  wh  ere  momen  tary  inability  to  respond  effectively 
to  an  unexpected  situation,  a  lapse  of  attention,  may  result  in 
serious  or  fatal  consequences,  it  is  important  to  know  more  about  the 
behavioral,  physiological  and  neurophysiological  signs  which  may 
preceed  such  states.  Such  knowledge  could  lead  to  new,  more  effective 
methods  of  detecting  the  precursors  of  the  impaired  condition,  b  e  f o  r  e 
any  impairment  is  evident  in  actual  performance. 
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Fig.  4.  Patterns  of  i n t e r - e I ec t r ode  covariances  for  Left  and  Right 
Cue  conditions  computed  from  “delta-band"  (.1  to  3  Hz  bandpass) 
filtered  data  of  7  persons  in  a  375  msec  wide  interval  centered  at 
687  msec  after  the  cue.  Patterns  for  right  cues  are  mostly  left¬ 
sided;  patterns  for  left  cues  are  mostly  right-sided.  The  midline 
anterior  central  area  is  a  focus  of  covariances  in  both  conditions, 
reflecting  activity  of  premotor  and  supplementary  motor  cortex.  Left 
frontal  covariances  are  prominent  for  both  hands. 


Fig.  5.  Stimulus-evoked  covariance  patterns  for  Left  and  Right  Move 
conditions  on  a  187  msec  wide  interval  centered  at  82  msec  after 
stimulus  onset.  Covariances  were  computed  from  "theta-band"  (4  to  7 
Hz  bandpass)  filtered  data.  At  this  interval,  covariances  show  a 
similar  pattern:  midline  posterior  parietal  electrode  covariances 
lead  midline  anterior  parietal,  which  lead  midi  ine  pr emo tor  electrode 
covariances.  Note  add i t i onal  covariances  from  the  right  posterior 
parietal  electroaes  for  the  r i gh t - r o t a t ed  foveal  stimuli.  These  may 
reflect  the  projection  to  the  scalp  from  generators  deep  in  the  left 
calcarine  fissure. 
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Fig.  6.  Stimulus-evoked  covariance  patterns  for  the  Left  and  Right 
Move  and  No-Move  trials.  Covariances  were  computed  from  "theta-band" 
(4  to  7  Hz  bandpass)  filtered  data  on  a  187  msec  wide  interval  cen¬ 
tered  at  312  msec  after  stimulus  onset.  The  midline  premotor  elec¬ 
trode  lags  the  left  and  right  prefrontal  electrodes*  as  well  as  the 
midline  prenatal  electrode  in  the  No-Move  trials.  This  may  be  a  sign 
of  motor  inhibition. 


Fig.  7.  Covariance  patterns  for  Left  and  Right  Move  trials,  computed 
from  "theta-band"  (4  to  7  Hz  bandpass)  filtered  data,  62  msec  after 
movement  onset.  At  the  peak  of  the  finger  pressure,  covariance  pat¬ 
terns  are  appropriately  contralateral  to  the  hand  used.  The  midline 
premotor  electrode,  which  is  the  focus  of  all  covariances,  leads  the 
prefrontal  electrodes  by  16-31  msec. 
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Fig.  8.  Covariance  patterns  for  accurate  and  inaccurate  feedback  in 
left-hand  trials  computed  from  “theta-band"  (4  to  7  Hz  bandpass) 
filtered  data  in  a  187  msec  wide  interval,  centered  375  msec  after 
feedback.  There  is  a  common  pattern  involving  covariances  from  mid¬ 
line  occipital  and  parietal  to  midline  premotor  and  right  prefrontal 
electrodes.  When  the  responses  were  inaccurate,  the  feedback  pattern 
additionally  included  the  left  prefrontal  electrode.  The  right  fron¬ 
tal  and  midi i ne  premotor  components  common  to  both  accurate  and  inac¬ 
curate  trials  may  reflect  “motor  recalibration,"  while  the  left  pre¬ 
frontal  component  in  inaccurate  trials  may  reflect  “updating"  in 
attent ional  set  for  behavioral  verification. 


Fig.  9.  Covariance  patterns  reflecting  prestimulus  differences 
between  accurate  and  inaccurate  trials,  computed  from  “delta-band" 
(.1  to  3  Hz  bandpass)  filtered  data  in  a  375  msec  wide  interval 
centered  687  msec  post  cue,  i.e.,  313  msec  p r e - s t i mu  I  us ,  Note  the 
greater  complexity  of  subsequently  accurate  trials.  Covariances 
overlying  left  prefrontal  cortex  are  prominent  in  accurate  trials, 
but  are  absent  in. inaccurate  trials.  This  may  reflect  the  involvement 
of  the  left  prefrontal  cortex  in  establishing  a  preparatory  set 
necessary  for  accurate  performance. 
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Only  a  very  few  studies  have  attempted  to  measure  behavioral ( 
physiological  and  neurophysiological  variables  during  controlled 
f a t i gue - i nduc i ng  conditions.  The  study  of  operational  fatigue  rests 
on  the  ability  to  produce  a  state  of  performance  decrement  in  an 
experimental  situation  specifically  controlled  for  the  factors  of 
initial  task  learning  and  subsequent  automatization  of  performance, 
so  that  the  effects  of  fatigue  can  be  distinguished  from  learning  and 
automatization.  The  recording  session  must  extend  from  a  state  of 
we  I  I -pract i ced .  alert  performance  to  later  stages  in  which  decrements 
of  performance  due  to  fatigue  are  clearly  evident  in  performance  mea¬ 
sures.  Also.  a  fol low-up  recording  in  an  unfatigued  state  must  be 
made  in  order  to  determine  whether  any  results  observed  in  the 
fatigued  state  were  actually  due  to  automatization  of  task  perfor¬ 
mance  over  many  hundreds  of  trials.  It  is  crucial  that  the  tasks  be 
sufficiently  taxing  in  the  alert  state,  yet  able  to  be  performed  in 
the  fatigued  state.  These  tasks  should  have  reasonable  construct  and 
face  validity  for  real-life  situations,  such  as  piloting  an  aircraft, 
and  must  meet  the  rigorous  requirements  of  neurophysiological  record- 
i ng  and  ana  lysis. 

2.  Exper i men  t  Deve I opmen  t .  Over  the  last  four  years,  an 
i n t e r d i sc i p I i nar y  team  of  researchers  from  four  organizations  collab¬ 
orated  on  designing  a  study  of  operational  fatigue.  The  group  con¬ 
sisted  of  James  Miller,  USAF  School  of  Aerospace  Medicine,  San 
Antonio;  Henry  Jex  and  James  Smith,  Systems  Technology  Inc.,  Los 
Angeles;  John  Stern,  Washington  University,  St.  Louis;  and  Alan 
Gevins  and  other  personnel  of  the  EEG  Systems  Laboratory. 

After  some  development,  a  suitable  paradigm  was  developed  which 
implemented  the  above  criteria.  In  February  1983,  a  pilot  recording 
was  made  at  the  EEG  Systems  Laboratory  involving  a  single  fatigue 
session  in  a  previously  practiced  subject.  It  consisted  of  four  repe¬ 
titions  of  the  performance  of  2-hour  modules  ("sorties'),  each  sortie 
containing  four  tasks.  Two  tasks  tested  rote  visuomoto r  performance, 
one  with  a  moderate  task  load  (subcritical  tracking),  the  other  with 
a  very  high  load  (critical  tracking).  The  other  two  tasks  placed  a 
high  load  on  short-term  memory  and  attention:  one  used  auditory  stim¬ 
uli,  and  the  other  used  visual  stimuli,  and  included  a  difficult 
motor  coordination  response  with  feedback.  Preliminary  analysis  of 
the  data  indicated  that  sustained  performance  of  the  tasks  induced 
both  behavioral  and  neurophys i o I g i ca I  signs  of  fatigue  (Interim 
Progress  Report,  1APR84,  Contract  F 3 36 1 5  -  8 2 - C -  0 6 2 8 ,  U.S.  School  of 
Aerospace  Medicine).  The  behavioral  measures  provided  a  clear  indi¬ 
cation  of  intervals  during  which  performance  was  impaired.  Examina¬ 
tion  of  averaged  even t  -  r e I  a t ed  potentials  indicated  that  in  the 
fatigued  state  the  amplitudes  of  the  N2  and  P3  peaks  were  reduced. 

Based  on  these  encouraging  results,  a  final  paradigm  was  implemented 
i n  wh ich  each  full  fatigue  recording  consisted  of  three  parts:  (1) 
practice  day  (4  two-hour  task  modules),  (2)  fatigue  day  (7-8  two-hour 
task  modules),  and  (3)  follow-up  day  to  me as u re  recovery  and  provide 
a  control  for  automatization  of  task  performance  (3  two-hour  task 
modules).  Fifty-one  brain  potential  channels,  two  eye-movement  chan¬ 
nels,  muscle  potentials,  respiration  and  electrocardiogram  were 
recorded.  During  the  last  half  of  1  3  84  and  the  f 1  r s  t  half  of  1385. 
full  "fatigue  recordings"  we  re  made  from  four  Air  Force  fighter  test 
pilots.  (Only  33  brain  potential  channels  were  recorded  during  the 
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fatigue  session  on  two  of  the  pilots,)  Highly  sensitive  types  of 
neurocognitive  pattern  analysis  (Gevins  e  t  a  I  • »  1981,  1983,  1984a, b, 
1985),  which  may  be  able  to  discern  predictive  indicators  of  impaired 
performance,  are  now  being  app I i ed  to  these  data. 

3.  Exper i men  t  a  I  Protoco I  .  Three  United  States  Air  Force  test 
pilots  based  at  Edwards  Air  Force  Base  in  California,  and  one  USAF 
test  pilot  based  at  Albuquerque,  N.M.  served  as  subjects  for  this 
study.  The  mean  age  of  the  group  was  34  years,  with  a  range  of  31  to 
39.  All  subjects  were  right-handed  males  in  excellent  health.  The 
experiment  was  divided  into  four  recording  sessions,  conducted  over 
the  course  of  four  days.  Session  One,  "Task  Acquisition,"  on  the 
first  day,  consisted  of  the  pilots  performing  the  experimental  tasks 
for  about  eight  hours,  unti  I  performance  asymptoted.  Sessions  Two  and 
Three  took  place  on  the  second  day.  During  Session  Two,  "Practiced 
Performance,"  pilots  performed  the  tasks  for  about  eight  hours,  and 
then  continued  into  Session  Three,  "Fatigue,"  It  consisted  of  another 
6-8  hours  of  recording,  and  terminated  between  12:25  and  3:20  a.m,, 
when  one  or  both  of  the  fol lowing  occurred: 

(a)  Subjects  asked  to  end  the  session. 

(b)  Useable  EEG  data  could  no  longer  be  recorded  because 
of  excessive  eye  movements  or  head  nodding  and  drooping. 

On  the  third  day  no  recording  was  done  and  subjects  were  allowed  to 
rest.  Session  Four,  "Over  I  earned , "  took  place  on  the  fourth  day,  and 
consisted  of  another  6-7  hours  of  recording  in  order  to  measure  prac¬ 
tice  effects, 

Recording  procedures  began  at  8:30  A.M.  Orientation,  electrode  appli¬ 
cation  and  position  measurement,  system  checkout  and  calibrations 
took  about  3  hours.  The  experiment  was  conducted  in  an  acoustically 
dampened,  a i r - cond  i  t  i oned  chamber;  room  temperature,  lighting,  and 
seating  position  were  adjusted  as  necessary.  Because  subjects  were 
seated  for  long  periods  during  each  session  (up  to  18  hours  on  Day 
2),  all  possible  efforts  were  made  to  ensure  their  continuing  com¬ 
fort.  Subjects  were  brought  water  and  permitted  to  stretch  at  regular 
intervals.  They  were  allowed  to  drink  coffee  or  soft  drinks  during 
the  day  and  with  dinner;  after  dinner  only  non - ca f f e i na t ed  drinks 
we  re  provided  for  all  but  one  subject.  The  experiment  consisted  of 
five  different  tasks:  (1)  the  V i suo -  Mo  tor  Monitoring  Task,  null  mem  - 
ory  condition  ( VMMT  0-back);  (2)  the  Visuo-Motor  Monitoring  Task, 
memory  load  condition  (VMMT  2-back);  (3)  an  Auditory  Monitoring  Mem¬ 
ory  (AUM)  Task;  and  the  (4)  Sub-critical  and  )5)  Critical  Tracking 
Tasks  (Allen  k  Jex,  1971;  Jex,  1978).  Recording  sessions  began  and 
ended  with  the  Tracking  Tasks.  In  between,  the  other  three  tasks  were 
organized  into  sorties,  consisting  of  100  to  300  trials  of  each  type. 
Table  1  shows  a  typical  sortie  for  sessions  1,  2  and  4.  Session  3, 
"Fatigue,"  differed  in  two  ways:  (1)  it  did  not  include  the  0-back 
VMMT  task,  and  z9?)  an  easier  version  of  the  VMMT  2-back  task  was 
used  during  the  later  part  of  the  session  when  the  subject's  perform¬ 
ance  was  declining  significantly. 

All  four  recording  sessions  began  and  ended  with  the  critical  and 
sub-critical  tracking  tasks  (CTT/SCT),  except  Session  3,  where  they 
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Tab  I e  ± 

Typical  “sortie"  for  sessions  11,  2  k  4 


VISUO-MOTOR  MONITORING  TASKS: 

NO  MEMORY  LOAD  (O-back),  200  TRIALS  (30  min.) 
MEMORY  LOAD  (2-back),  300  TRIALS  (45  min.) 

AUDITORY  MONITORING  TASK,  150  TRIALS  (15  min.) 
TRACKING  TASKS  (15  min. ) 
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were  also  included  in  the  middle.  In  between  the  tracking  tasks,  the 
VMMT  0-back,  VMMT  2-back,  and  the  AUM  tasks  alternated.  For  these 
three  tasks,  sorties  consisted  of  two  or  four  50-trial  blocks  for  the 
0-back  (null  memory)  task,  three  or  six  blocks  of  50  for  the  2-back 

(memory  load)  task,  and  three  blocks  of  50  for  the  AUM  task.  Each 

block  was  se I f - i n i t i a t ed  by  the  participant. 

The  AUM  task  consisted  of  one  memory  condition:  three-back  memory 
load.  In  this  task,  the  pilots  were  asked  to  respond  whenever  the 
current  stimulus  was  the  s ame  as  the  number  three  back,  Stimuli 
consisted  of  single-digit  numbers  from  0  to  9  generated  by  a  Votra' 
speech  synthesizer  (duration  from  245  to  430  msec),  and  presented  in 
blocks  of  50.  The  i n t e r s t i mu  I  us  interval  varied  randomly  from  4  to  5 
seconds.  Stimul i  were  presented  over  two  speakers,  located  in  front 
of  the  pi  lots;  as  in  the  VMMT,  pi  lots  responded  by  pressing  on  a 
force  transducer  with  the  forefinger  of  their  right  hand.  In  the  AUM 

task,  unlike  the  VMMT,  the  response  registered  as  either  on  or  off, 

and  there  was  no  feedback  or  warning  signal  for  individual  trials. 
Subjects  did  receive  feedback  at  the  end  of  each  block  of  trials, 
show i ng : 

(a)  Targets  ( ca t ch - t r i a  I s )  hit. 

(b)  Targets  missed. 

( c  )  False  pos i  t i ves . 

(d)  Average  reaction  time. 

(e)  Standard  deviation  of  reaction  time. 

The  tracking  tasks  were  developed  for  research  to  measure  psychomotor 
skills  vital  to  a  broad  class  of  tasks  likely  to  be  performed  in 
space  (Jex,  1978;  Allen  k  Jex,  1971),  Both  the  Critical  and  Sub  - 
critical  tracking  tasks  required  the  pilot  to  control  the  motion  of  a 
luminous  horizontal  CRT  line  with  an  isometric  (force)  control  stick. 
As  the  d  y  nam ically  unstable  horizontal  line  fell  off  fr  om  a  reference 
point  in  the  center  of  the  screen,  the  pilot  applied  compensatory 
pressure  on  the  control  stick.  Difficulty  in  the  SCT  task  w a s  held 
constant  at  a  level  wh  ich  the  pilot  could  ma  intam  for  10  minutes,  in 
the  CTT  the  instability  of  the  line  steadily  increased  at  a  pace 
which  brought  loss  of  control  in  one  to  two  minutes.  CTT  trials  ended 
when  control  was  lost.  Ten  minutes  of  SCT  we  re  foil  owed  by  five  CTT 
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eye  movemen  t  s . 
used  previously 
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background  of 
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force  transduce 


a i  I ed  Descr  ipt  ion  o  f  VMM  T  Task.  The  VMMT  was  des  i gned 
se  con  t  r  o I  of  s  t  i mu  I  us  par  ame  t  e  r  s ,  mo  tor  activity  and 
This  was  done  by  combining  the  numeric  judgment  task 
(Gevins  et  a  I  .  ,  1931)  wi  ih  a  memo  r  y  el emen  t  based  on 

igit  cancel1 in g  task.  Stimuli  consisted  of  single- 
1325  msec  duration)  presented  on  3  Videographics-  I  I 
or  70  cm  from  the  subject.  Stimuli  subtended  a  visual 
degrees,  with  an  illumination  of  0.5  i :g  tL  against  a 
-1.5  log  f  L .  Subjects  responded  c r  each  t  r  , a l  with  3 
action  of  their  fight  ■  n de  <  finger  on  an  .  5  ome  t  r  •  ; 
r  with  a  force  proportional  to  the  stimulus  number  c  n 
from  1  to  3.  A  2  - d 1  a  1  t  feedback  number  1  3  2  5  msec 
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duration)  indicating  the  exact  force  applied  to  one-tenth  of  a  unit 
was  presented  1  sec  after  peak  response  pressure.  The  inter-trial 
interval  was  1.8  sec,  during  which  a  fixation  symbol  (X)  was  on  the 
screen.  To  rule  out  the  possibility  of  memorization  of  stimulus 
sequences  across  the  series  of  recording  sessions.  5  different  series 
of  150  stimuli  for  the  2-back  and  5  series  of  100  stimuli  for  the 
0-back  task  were  formed  from  r andom- number  tables.  In  order  to  disso¬ 
ciate  motor  and  memory  errors  in  move  trials  of  the  2-back  memory 
task.  the  stimulus  numbers  were  chosen  so  that  the  two  numbers  held 
i n  memo  ry  and  the  current  sti mu  I u  s  number  we  re  all  at  least  two 
digits  apart  (e.g.,  5-7-3).  (The  Fatigue  session  used  different 

sequences  where  the  numbers  were  at  least  three  digits  apart  because 
of  the  degraded  level  of  motor  control.) 

The  feedback  number  was  underlined  if  the  response  was  sufficiently 
accurate.  Accuracy  was  based  on  an  adaptive  error  tolerance,  the  geo¬ 
metric  average  of  the  error  (the  distance  from  the  required  response 
pressure)  on  the  previous  5  move  trials  (Gevins  et  a  I  . ,  1981.  1983). 

Move  trials  falling  within  the  adaptive  error  tolerance  were  termed 
"win"  trials.  This  adaptive  error  tolerance  served  to  equalize  task 
difficulty  across  the  session  and  to  indicate  the  current  performance 
trend.  The  timing  of  a  2-back  memory  trial  is  illustrated  in  Table  2. 
After  each  50-trial  block,  summary  performance  statistics  were  pre¬ 
sented  on  the  video  screen.  These  were: 

(a)  Average  movement  force  error  (difference  from 
required  pressure)  and  its  standard  deviation. 

(b)  Average  adaptive  error  tolerance  (running  mean  of 
previous  five  trials)  and  its  standard  deviation. 

(c)  Average  response  time  and  its  standard  deviation. 

<d)  The  number  of  target  trials  (catch  trials)  presented, 
and  the  number  missed  (false  positives). 

(e)  Number  of  time  outs  (no  response  within  set  period). 

The  two  memo  ry  load  conditions  in  the  VMMT  we  re:  a  null  memo  r  y  or 
0-back  condition,  in  which  the  required  response  pressure  was  indi¬ 
cated  by  the  current  stimulus  number;  and  a  2-back  memory  condition, 
in  which  the  response  pressure  was  indicated  by  the  stimulus  number 
two  trials  back.  In  order  to  add  a  recognition  element  to  the  tasks, 
and  also  to  avoid  overly  short  response  times  in  the  memory  task,  20% 
of  the  trials  in  both  conditions  were  designated  as  catch-tr lals; 
subjects  were  required  not  to  respond  during  those  trials.  As  shown 
in  Table  3,  in  the  0-back  condition,  these  no-move  catch  trials  were 
those  where  the  stimulus  number  was  0;  in  the  2-back  condition  (Table 
4),  catch  trials  were  designated  as  the  occurrence  of  a  stimulus  num¬ 
ber  identical  to  the  2-back  stimulus  number. 

The  difference  between  the  0-back  and  2-back  tasks  was  the  use  of 
short-term  working  memory  to  maintain  the  two  constantly  changing 
numbers  and  recognition  of  the  2-back  identical  numbers  which  were 
the  no-move  "targets'  in  the  memory  task,  in  contrast  to  the  imme¬ 
diate  response  to  current  stimulus  numbers  and  recognition  of  the 
number  'O'  which  was  the  on  I )  no-move  target  in  the  non-memory  task. 


Table  ± 

2-BACK  (MEMORY)  VMMT  TASK:  Typical  sequence  of  6  trials  at  beginning 
®  of  block.  "N"  indicates  no  response  movement  (catch  trial). 


STIMULUS: 

3 

7 

6 

9 

6 

2 

RESPONSE: 

N 

N 

3 . 7 

7 . 2 

N 

8.9 

FEEDBACK : 

OK 

OK 

3  .  7 

7 . 2 

OK 

8 . 9 

28 
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It  is  assumed*  of  course,  that  accurate  ongoing  motor  performance  in 
both  tasks  required  maintenance  of  a  motor  "memory*  and  its  adjust¬ 
ment  in  response  to  feedback.  In  the  memory  task,  except  for  the 
first  2  trials  of  a  block  where  no  response  was  required,  each  trial 
required:  (1)  judgement  of  whether  the  current  stimulus  number 
matched  the  2-back  number  and  withholding  response  if  it  was  (no-move 
trials,  quasi-random  20%);  (2)  execution  of  a  precise  pressure  corre¬ 
sponding  to  the  2-back  number  if  it  did  not  match  the  current  stim¬ 
ulus  (move  trials,  80%);  (3)  deletion  of  the  2-back  number  from  mem¬ 
ory,  and  maintenance  of  the  1-back  number,  storage  of  the  current 
stimulus  number  and  their  order;  and  (4)  evaluation  of  the  feedback 
number  in  order  to  maintain  accurate  motor  performance.  The  latter 
operation  could  involve  interference  between  the  stored  numbers  and 
the  feedback  number.  Trials  averaged  about  B  seconds  in  length;  the 
time  between  presentation  of  a  stimulus  number  and  response  to  that 
number  in  the  memory  task  was  about  12  seconds.  It  was  found  that 
this  pace  was  fast  enough  to  al low  maintenance  of  the  numbers  in 
short-term  memory  without  rehearsal.  Participants  commonly  reported 
that  they  developed  a  "sense"  for  the  2-back  number  (although  one 
person  reported  a  visual  strategy)  sufficiently  accurate  to  permit 
response  pressures  to  be  almost  as  accurate  as  the  non-memory  task. 
Also,  some  persons  developed  faster  response  times  in  the  memory  task 
during  the  course  of  the  recording  sessions,  yet  exhibited  no  defici¬ 
ency  in  detection  of  no-move  stimulus  matches.  This  suggests  a  prim¬ 
ing  of  the  motor  system  (since  the  relevent  2-back  number  is  known), 
with  the  ablity  to  abort  this  preparatory  facilitation  upon  recogni¬ 
tion  of  a  2-back  match. 

5.  Recording  Methods.  Fifty-one  EEG  channels,  referenced  to 
A2 ,  were  recorded.  The  montage  is  shown  in  Figure  10.  Recordings  of 
the  first  two  subjects,  made  during  the  transition  period  from  26  to 
51  channels,  were  made  with  a  33-channel  montage  which  had  sparser 
coverage  over  the  top  of  the  head  and  fewer  electrodes  at  the  per i - 
phery.  Vertical  and  horizontal  eye  movements  from  electrodes  above 
and  below  one  orbit  and  at  outer  canthi  were  also  recorded,  as  were 
the  responding  f I e  x  o  r  d  i  a  i  tor i  muscle  potentials  and  output  of  the 
response  force  transducer.  Al,  EKG  and  respiration  were  also  re¬ 
corded.  Brain  potentials  were  recorded  with  an  Electrocap  Interna¬ 
tional  nylon  mesh  cap  using  tin  electrodes  and  SnC I  2  electrolyte  gel, 
amplified  by  a  64-channel  Bioelectric  Systems  Model  AS-64P  with  .016 
to  50  Hz  passband  and  digitized  to  11  bits  at  128  Hz.  All  signals 
were  monitored  on  3  8-channel  and  2  16-channel  polygraphs. 

Table  5  gives  an  overall  picture  of  a  typical  recording.  Table  6 
shows  1 n  mor  e  detail  the  second,  or  fatigue,  day,  including  the 
pilots'  self  report  of  fatigue,  which  was  requested  periodically. 
Subjective  Fatigue  was  measured  on  a  7-point  scale  (Crew  Status 
Survey  ,  USAF  SAM-  202  )  . 

6.  Format  ion  o  f  Data  Sets.  Polygraph  records  were  edited 
off-line  by  t wo  independent  raters  to  eliminate  trials  with  evidence 
of  eye  movement  in  the  EOG  channels  and  muscle  or  instrumental  arti¬ 
facts  in  the  EEG  channels  from  three-quarters  of  a  second  before  the 
stimulus  to  one  and  one-quarter  seconds  after  the  end  of  the  feed¬ 
back.  Move  trials  where  response  movement  was  not  unimodal  and  bal¬ 
listic,  or  not  made  within  1.25  seconds  after  the  stimulus,  we r e 
eliminated.  No-move  trials  where  mo v  erne n  t  occurred  we  re  eliminated. 
Data  sets  were  pruned  of  outliers  for  EOG,  EMG  and  behavioral  var  1  - 
ables  such  as  move  velocity  and  acceleration. 


Table  5. 

FATIGUE  EXPERIMENT 
PILOT:  HW 


DATE 

SE3S 1  ON 

TIMES 

OF  DAY 

DURAT 1  ON 
(HOURS) 

23 

APR 

85 

1 

(LEARNING) 

11:25 

-  19:15 

7  :  50 

24 

APR 

85 

2 

( PRACT 1 CED ) 

11:25 

-  19:10 

7  :  45 

24 

APR 

85 

3 

( FAT  1 GUED ) 

20  :  20 

-  02:25 

6  :  05 

26 

APR 

85 

4 

( OVER-LEARNED ) 

12:10 

-  18:05 

5  :  55 

3  1 
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Tab  I e  & 

•  FATIGUE  DAY 

24  APR  85 

PILOT:  HW 


TIME  OF  DAY 

ELAPSED  TIME 

SUBJECT  1 VE 

(HOURS) 

FAT  1 GUE 

SESSION  2 
Sortie  1 

11:25 

-  13:15 

1:50 

3 

LUNCH 

Sortie  2 

14:25 

-  15:35 

4:10 

3 . 5 

• 

Sortie  3 

15:44 

-  17:20 

5  :  55 

4 . 5 

Sortie  4 

17:33 

-  19:10 

7  :  45 

5 

D  1  NNER 

SESSION  3 

• 

Sortie  1 

20  :  20 

-  22:55 

11:30 

5 . 5 

Sortie  2 

23:01 

-  00:15 

12:50 

7 

Sortie  3 

00:23 

-  01:25 

14:00 

7 

Sortie  4 

0  1:29 

-  02:25 

15:00 

7 

€ 


bession  3  terminated  when  participant  began  tailing  asleep. 
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Fatigued  and  rested  data  were  selected  based  on  reaction  time  and 
error.  For  Pilot  HW,  nine  blocks  of  2-back  data  (450  trials)* 
recorded  between  12:41  and  02:25  and  having  the  highest  error  and 
longest  reaction  times,  were  selected  as  representative  of  the 
fatigued  state.  Nine  blocks  with  the  lowest  mean  reaction  time  and 
error,  recorded  between  14:25  and  16:30  were  chosen  from  Session  2 
for  alert  data.  After  editing,  there  were  380  trials  from  Session  3, 
and  273  trials  from  Session  2.  As  shown  in  Fig.  11,  performance 
degraded  consistently  as  participants  fatigued.  Reaction  time  and 
error  increased,  even  as  trials  became  subjectively  easier  to  perform 
due  to  the  adaptive  error  tolerance  algorithm. 

7.  Prel  iminary  Ana  I  vs i s .  Prel  iminary  analyses  have  been  com¬ 
pleted  on  the  practiced  data  of  three  pilots  and  the  fatigued  data  of 
two  pilots.  In  the  alert  condition,  in  a  pre-stimulus  anticipatory 
interval,  delta-band  filtered  data  revealed  striking,  localized 
differences  between  conditions  with  a  low  and  high  load  on  short-term 
working  memory.  In  a  375-msec  wide  interval  centered  375  msec  before 
the  stimulus,  strong  covariances  between  a  left  anterior  frontal  and 
left  superior  t empo ral,  right  anterior  frontal  and*  right  pr emo tor 
electrodes  distinguished  the  high  memory  load  from  the  low  memory 
load.  In  the  pre-st imulus,  high-memory  load  interval  of  the  fatigue 
condition,  the  covar i ances  were  reduced  in  number  and  greatly 
weakened . 


V.  RECENT  DEVELOPMENTS  IN  NCP  ANALYSIS 
A .  Data  Col  lection 


1  .  64  -  channe I  EEG  Record i no  T  echn i que  and  Au  t  oma  ted  Artifact 
Re  i  ec  t ion.  Recording  capacity  has  recently  been  expanded  to  a  full  64 
EEG  channel  capacity  to  provide  uniform  scalp  coverage  with  an  inter¬ 
electrode  distance  of  about  3.25  cm  (Fig,  12),  All  recordings  are 
referenced  to  one  mastoid  (usually  A2 ) ;  the  other  (A1)  is  recorded  as 
a  separate  channel  so  that  a  lateral ly  balanced  (A1+A2)  reference  may 
be  compu  ted  off-line,  avoiding  the  undesirable  boundary  condition  on 
the  potential  field  created  by  linked-ear  references.  Signals  are 
digitized  at  12  bits  full  scale  at  256  samples/sec,  and  stored  on 
magnetic  tape  by  the  PDP11-60.  Forty-eight  of  the  64  channels  are 
monitored  at  a  time  for  electrode  problems.  Improved  automated  arti¬ 
fact  rejection  algorithms  using  statistical  pattern  recognition  to 
combine  amplitude,  frequency,  and  topographic  features  (Morgan  k 
Gavins,  In  prep.)  are  applied  off-line.  These  algorithms  satisfac- 
toral ly  perform  a  preliminary  data  screening. 
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24  APR  85 

FATIGUE  EXPERIMENT  pilot:  h.w. 


Fig.  11.  Performance  degradation  shown  as  participants  fatigued 


EES  SYSTEMS  LABORATORY 


77\ 


rs 


*kSi 


t  \ 


if  T  T  T 

r  •  •  T*  > 


S  **  3  v 

\  y/ 


Fig.  12(a).  Expanded  10-20  System  of  electrode  position  nomenclature 
Additional  coronal  rows  of  electrodes*  interpolated  between  the  In 
ternat ional  10-20  System  coronal  rows*  have  the  letter  a  for  'anter 
ior"  added  to  the  designation  for  the  next  row  anterior,  e.g..  aP 
for  anterior  parietal  midline  electrode.  With  64  electrodes,  th 
average  distance  between  electrodes  on  an  adult  head  is  ^3.25  cm. 
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Statistical  pattern  classification  techniques  were  used  to  evaluate 
the  performance  of  two  eye  movement  artifact  removal  techniques: 
Amplitude  Subtraction  Eye  Movement  Correction  Procedure  and  Spectral 
Subtraction  Eye  Movement  Correction  Procedure  (Fig.  13).  Although 
visual  compar i son  showed  both  methods  to  be  effective,  the  objective 
evaluation  technique  suggested  that  the  f requency - dependen t  system 
identification  method  may  some  t i mes  be  mo  re  effective  for  eye  blink 
r  emova I  . 

2.  Digitization  o  f  Electrode  Pos i  t ions.  Before  and  after  a 
recording  session,  the  position  of  the  external  center  of  the  grommet 
of  each  electrode  is  measured  in  three  dimensions  using  a  3-D  digi¬ 
tizer  (Fig.  14).  The  participant  rests  his  head  in  a  padded  stabil- 
i  z  i  rig  jig  for  the  5  minutes  required  to  digitize  the  positions  of  all 
electrodes.  Correction  to  scalp  positions  is  made  by  a  I eas t - squa r es 
fit  Multiple  Linear  Regression,  which  yields  the  general  ellipsoid 
surface  best  fitted  to  the  set  of  digitized  positions,  and  the  digi¬ 
tized  coordinates  are  translated  to  a  coordinate  system  centered  on 
this  ellipsoid. 

3 .  MR  I  I  mag i nq  for  Determining  Post  t  ions  o  f  Elect  r  odes  and 
Co  r  t  i ca I  Structures.  Full  sets  of  Magnet  ic  Resonance  Images  ( MR  I  ; 
Fig.  15)  are  made  at  UCSF  Long  Hospital  facility.  These  are  high- 
resolut ion  (0.8  mm  pixel)  3-axis  cross-section  images  of  soft  tissue 
1  cm  apart  over  the  whole  volume  of  the  head.  The  pictures  are  digi¬ 
tized  to  give  coordinate  surfaces  for  scalp,  outer  and  inner  skull 
surfaces,  and  cerebral  surface,  including  loci  of  major  f'ssures. 
Comparison  of  electrode  positions  and  MR  I  measurements  provides  cor¬ 
relation  of  electrode  positions  and  cortical  areas,  allowing  compari¬ 
son  of  functional  neuroanatomy  across  individual  participants  for 
localized  sensory  and  motor  cortical  ERP  componen  ts.  The  direct  mea¬ 
surements  of  thickness  of  scalp  and  skul I  provides  accurate  informa¬ 
tion  for  use  in  spatial  deconvolution  (see  below). 

B .  Data  Analysis 


1  •  Trial  Select  ion  Using  Pat  tern  Recogn i  t  ion .  A  great  deal 
of  effort  has  been  expended  in  attempts  to  form  improved  estimates  of 
the  averaged  ERP  (revi ews  in  McGillem  et  a  I  •  ,  1981,  1985;  McGill em  k 
Aunon,  in  press;  de  Weerd,  1981;  Gevins,  1984),  Most  methods  assume 
that  task-related  signals  are  present  in  every  trial  and  also  have 
inherent  assumptions  about  the  statistical  properties  of  signal  and 
noise.  We  have  developed  a  method  for  ERP  estimation  without  the 
first  assumption  and  with  a  relaxed  second  assumption.  To  do  this, 
separate  averages  are  formed  from  trials  which  are  correctly  or 
incorrectly  classified  by  a  statistical  pattern  recognition  pro¬ 
cedure.  In  the  averages  of  correctly  classified  trials,  the  ERP  peaks 
are  enhanced  in  comparison  with  the  original  averages.  The  averages 
of  incorrectly  classified  trials  resemble  the  background  EEG. 
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Several  dozen  applications  of  this  technique  suggest  that  it  is  suc¬ 
cessful  in  identifying  trials  with  a  poor  s i gna I  -  to- no i se  ratio*  and 
provides  a  simple  method  of  ERP  estimation  with  minimal  a,  priori 
assumptions  about  the  characteristics  of  the  signal. 

2.  W i one  r  ( T i me  - F  r  eouenc v )  Distributions.  The  ERP  waveform 
is  a  function  of  time  and  does  not  provide  explicit  frequency  infor¬ 
mation.  Power  spectra  of  ERP  waveforms  provide  frequency  information 
but  obscure  t i me - dependen t  phenomena.  A  view  of  the  spectrum  as  it 
changes  over  time  would  give  a  new  view  of  the  evolution  of  different 
frequency  components  of  the  ERP.  A  simple  approach  would  be  to  com¬ 
pute  the  spectrum  over  highly  overlapped  windows  of  the  average  ERP. 
However,  such  a  ’ spec t r og r am "  would  smear  together  events  within  each 
analysis  window.  Wh i le  there  is  an  unavoidable  trade-off  between  fre¬ 
quency  and  time  resolution  (known  as  the  Heisenberg  Uncertainty  Prin¬ 
ciple).  the  spectrogram  is  an  inflexible  way  of  determining  this 
choice. 

A  preferred  method  is  to  compute  a  general  function  of  time  and  fre¬ 
quency  called  the  Wigner  Distribution  (Claasen  k  Meek lenbrauker . 
1980;  Janze  k  Kaizer,  1983)  which  approximates  the  instantaneous 
energy  for  a  given  time  and  frequency,  In  practice,  the  "purified* 
ERPs  (obtained  using  the  procedure  above)  show  strong  enough  energy 
"peaks*  in  the  Wigner  Distribution  to  validate  very  simple  interpre¬ 
tations  of  the  time  and  frequency  locations  of  signal  energy.  The 
Wigner  Distributions  are  being  used  to  determine  digital  filter  char¬ 
acteristics  which  produce  optimal  t i me - f r equency  resolution  for  a 
given  data  set, 

3.  8a  I anc i nq  Data  Sets.  Special  attention  is  given  to  the 
preparation  of  data  sets.  since  we  believe  that  this  is  a  highly 
neglected  aspect  of  experimental  control.  In  order  to  form  balanced 
pairs  of  data  sets  for  each  neurophysiological  hypothesis,  the  total 
set  of  ar t i f ac t - f r ee  trials  from  each  recording  is  submitted  to  an 
interactive  program  ( AD  I  SORT )  which  displays  the  means,  t-tests  and 
histogram  distributions  of  about  50  behavioral  and  other  variables. 
Pairs  of  data  sets  can  be  quickly  inspected  for  significant  differ¬ 
ences  in  variables  not  related  to  the  hypothesis  and  pruned  of  out¬ 
lier  trials  until  balanced,  usually  to  a  conservative  alpha  of  .2. 
The  AD  I  SORT  program  greatly  facilitates  the  handling  of  large  amounts 
of  data  and  permits  a  high  degree  of  control  of  stimulus,  behavioral 
and  other  variables  which  would  not  otherwise  be  possible.  It  outputs 
labeled  lists  of  trials  which  vary  on  I y  according  to  the  chosen 
hypothesis  to  be  submitted  to  the  signal  processing  and  analysis 
programs  of  our  system.  Variables  include  stimulus  parameters, 
responding  finger,  response  onset  and  i n t e r -mo vemen t  times,  response 
movement  force,  velocity,  acceleration  and  duration,  error  and  ada p- 
t i ve  performance  measure.  There  are  also  indices  of  eye  movement,  EMG 
activity  and  'arousal,'  which  are  measures  of  integrated  energy  in 
the  vertical  and  horizontal  EOG  channels,  the  EMG  channel  and  the  Pz 
electrode,  respectively,  computed  in  separate  500-msec  epochs  before 
and  after  the  onset  of  cue,  stimulus,  response  and  feedback. 

4 .  EEG  Spatial  Signal  Enhancemen  t .  Electrical  currents  gen¬ 
erated  by  sources  ;n  the  brain  are  volume  conducted  through  brain, 
ce r eb r osp i na I  fluid,  skull  and  scalp  to  the  recording  electrodes. 
Because  of  this,  potentials  due  to  a  localized  source  are  spread  over 
a  considerable  area  of  scalp,  so  that  the  potential  measured  at  a 
scalp  site  represents  the  summation  of  signals  from  many  sources  over 
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much  of  the  brain.  (We  have  estimated  the  'point  spread’  for  a  radial 
dipole  in  the  cortex  to  be  about  2.5  cm  Fig.  16).  This  spatial 
low-pass  filtering  makes  source  localization  difficult!  even  for  cor¬ 
tical  sources!  and  causes  the  potentials  from  local  sources  to  be 
mixed  with  those  from  more  distant  generators.  By  modeling  the  resis¬ 
tive  properties  of  the  tissue  between  brain  and  scalp  surface!  we  can 
perform  a  deconvolution  of  the  potential  to  the  level  of  the  brain 
surface  without  imposing  assumptions  as  to  the  actual  (cortical  or 
sub -cor t i ca I )  source  locations.  This  technique  is  being  evaluated! 
along  with  the  local  Lap  I ac i an  operator  (or  "Hjorth  montage*). 

(a)  Lap  I ac i an  Operator .  Poisson's  equation  states  that 
the  Lap  I ac i an  of  the  potential  is  proportional  to  the  local  density 
of  current  sources.  An  estimate  of  the  average  Lap  I ac i an  over  a  vol¬ 
ume  of  scalp  surrounding  a  given  electrode  will  then  be  proportional 
to  the  net  current  leaving  that  volume.  The  2 - d i mens i ona I  Lap  I ac i an 
in  the  plane  of  the  scalp  is  proportional  to  the  net  current  through 
the  edges  of  the  volume.  Since  there  are  no  current  sources  in  the 
volume.  this  net  current  is  equal  to  the  local  density  of  current 
flow  up  through  the  skull  (local  current  source  density  or  CSD ) 

(Hjorth  1975.  1980.  1982;  Thickbroom  et  a!..  1984  -  Fig.  17b), 

[Note  that  this  term  does  not  refer  to  neuronal  sources,  but  only  to 
local  current  emerging  from  the  skull.] 

This  transform  has  a  localizing  effect  since  it  is  relatively  insen¬ 
sitive  to  sources  beyond  the  proximal  electrodes,  and  has  the  advan¬ 
tage  of  being  independent  of  reference  electrode  site.  The  usual 
implementation  (Hjorth,  1980;  Thickbroom  et  al . .  1984)  estimates  the 
Laplacian  operator  at  a  given  electrode  k  as  an  average  of  the  dir¬ 
ectional  derivatives  from  each  surrounding  electrode  i  to  the  given 
e I ec  t  r ode  k  ; 

iVk  (vk-vi>/Rik 
Lk  =  - 

£  i/ 

i^k  / 

where  vi  are  the.  potentials  at  the  respective  electrodes  and  Rik  are 
the  distances  separating  the  electrodes.  This  quantity  is  propor¬ 
tional  to  a  second - d i f f e r ence  estimate  of  the  Laplacian  for  a  'ro¬ 
sette'  of  electrodes  uniformly  surrounding  the  given  electrode 
(Hjorth,  1975),  with  the  factor  of  propor t i ona I i t y  being  a  geomet¬ 
rical  factor  (the  electrode  separation  squared),  but  departs  from  the 
same  proportionality  as  the  configuration  of  electrodes  departs  from 
the  symmetrical,  notably  when  all  the  proximal  electrodes  are  on  one 
side  of  the  given  electrode.  We  have  improved  upon  the  conventional 
method  of  Hjorth  (1975)  by  using  actual  measured  interelectrode  dis¬ 
tances.  However,  we  have  been  unable  to  overcome  the  difficulties  as¬ 
sociated  with  the  Laplacian  computation  for  peripheral  channels,  and 
do  not  include  them  in  analysis. 

( b )  Spatial  Deconvolution  for  Reduc i nq  the  Effect  o f 
Vo  I ume  Conduct  ion .  Spatial  deconvolution  is  the  mode  ling  of  the  prop¬ 
agation  of  electric  potentials  from  brain  to  scalp.  The  simplest 
analytic  deconvolution  approach  is  to  represent  the  tissue  between 
brain  and  scalp  electrodes  as  concentric  spheres  with  different 
resistivities  (Fig.  18).  The  form  of  the  potential  wherever 
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Fig.  17.  Three  representat ions  of  averaged  st imulus- locked  event- 
related  waveforms  for  visual  move  trials  from  participant  AV07,  Fifty 
channels,  (a)  Common  Average  Reference,  (b)  Laplacian  (second  spatial 
derivative  of  potential  field). 
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there  are  no  sources  is  represented  as  the  general  solution  of 
Laplace's  equation  in  spherical  polar  coordinates: 


where  the  spherical  harmonic  functions  Ynm(® >4>)  which  contain 
all  the  angular  dependence  of  the  potential*  are  solutions  of 
Laplace's  equation  forming  a  complete  orthonormal  basis  on  the  unit 
sphere*  and  the  A's  and  B's  are  coefficients  to  be  determined  from 
the  boundary  conditions*  At  each  interface  of  different  media*  the 
potential  and  the  current  normal  to  the  surface  must  be  continuous: 


where  is  the  radius  of  the  boundary  between  media  i  and  i ♦ 1 ,  and 
where  the  are  the  conductivities  of  the  respective  media*  These 
boundary  conditions  constitute  a  series  of  equations  which  may  be 
solved  in  several  ways:  to  represent  the  coefficients  in  the  expan¬ 
sion  of  the  electrical  potential  at  the  surface  of  the  scalp,  in 
terms  of  the  coefficients  in  the  expansion  at  the  surface  of  the 
brain,  or  in  terms  of  a  distribution  of  dipole  sources  at  some  level 
in  the  cortex.  The  potential  at  the  scalp  is  then  represented  as  a 
sum  (convolution)  over  cortical  sources  or  potential  distributions, 
and  inversion  of  this  relation  (deconvolution)  allows  one  to  rep¬ 
resent  the  sources  (cortical  potential  coe f f i c i en t s )  in  terms  of  the 
measured  potentials. 

This  procedure  is  independent  of  hypotheses  about  the  character, 
position,  or  orientation  of  the  neuronal  sources  of  electric  poten¬ 
tial  other  than  that  the  tissue  intervening  between  brain  and  scalp 
are  free  of  any  active  sources.  In  practice,  the  thicknesses  of  scalp 
and  skull  at  each  electrode  position  are  measured  from  the  MR  I  images 
of  each  person,  The  CSF  shell  has  been  omitted  in  our  initial 
studies,  following  the  results  of  the  Cal  Tech  Group  (Ary  et  a!., 
1981;  Kavanagh  et  al.,  1978;  Darcey  et  al.,  1980).  We  are  experiment¬ 
ing  with  methods  of  measuring  the  conductivities  of  the  layers. 

Several  recordings  of  an  aud i tor y - v i sua I  ( AV )  task  were  made  with  50 
scalp  channels  during  the  summer  of  1989.  The  positions  of  electrodes 
were  digitized  using  a  s t er eopho t ogr amme t r y  system  developed  for 
craniofacial  research  (Baumrind  fc  Curry,  1984;  Curry  et  al.,  1982). 
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(This  has  since  been  replaced  by  direct  digitization  of  the  electrode 
locations.)  The  digitized  positions  were  used  to  implement  the  trans¬ 
formation  by  the  Laplacian  operator  and  a  form  of  the  spatial  decon¬ 
volution  modeling  in  terms  of  a  distribution  of  radial  dipoles  over 
the  cortex  in  a  4-sphere  model  of  the  head  (cerebrum*  cer eb r o - sp i na I 
fluid*  skull*  and  scalp). 

The  Laplacian  estimate  at  each  electrode  was  computed  as  the  average 
of  the  directional  derivatives  from  adjacent  electrodes  (Figs.  17b  k 
19b).  The  deconvolution  model  represents  the  sea  I p - r eco r ded  potential 
distribution  as  deriving  from  a  distribution  of  radial  dipoles  in  the 
cortex  (Figs.  17c  k  19c).  In  order  to  assess  the  reduction  in  inter¬ 
electrode  correlation  due  to  volume  conduction,  we  have  calculated 
the  i n t er e I ec t r ode  correlation  over  the  s t i mu  I  us  -  I ocked  visual  ERP 
for  each  pair  of  channels  for  the  common  average  reference,  Lapla¬ 
cian,  and  deconvolved  ERPs.  Then,  using  the  measured  electrode  posi¬ 
tions  and  the  4-sphere  model  of  the  head,  we  calculated  the  point 
spread  for  each  pair  of  electrodes.  A  linear  regression  of  correla¬ 
tion  versus  point  spread  reveals  the  degree  of  association  for  the 
common  average  derivation.  The  same  regression  for  the  Laplacian  and 
deconvolved  derivations  shows  the  degree  of  association  to  be  more 
reduced  for  the  deconvolution  than  for  the  Laplacian  (Table  7). 

The  spherical  model  of  the  head  is  not,  of  course,  a  realistic  rep- 
resentat i n  of  the  true  geometry  of  an  individual's  head  and  brain. 
The  temporal  areas  are  particularly  problematic  because  of  their 
relative  flatness.  We  are  formulating  a  more  realistic  model  using 
finite-difference  representations  of  the  electrical  properties  of  the 
tissue  intervening  between  the  cortex  and  the  surface  and  the  scalp. 
The  key  to  this  method  is  the  availability  of  structural  information 
about  each  participant's  brain,  skull  and  scalp  from  the  Magnetic 
Resonance  Image  ( MR  I  )  . 

5.  Source  Loca I i zat i on .  Given  an  adequate  model  of  the 
tissue  of  the  head,  a  knowledge  of  the  positions,  multipole  charac¬ 
ter,  orientations,  and  magnitudes  of  the  sources  of  the  brain  elec¬ 
trical  potential  and  magnetic  fields  suffices  to  determine  the 
measured  potentials  and  fields  at  the  scalp  (Plonsey,  1969;  Nunez, 
1981).  Since  the  potentials  may  be  considered  quasi-static  (Plonsey, 
1969),  the  spread  of  potential  is  governed  by  Poisson's  equation, 

V  •  (a  V  $  )  =  lv 

where  $  is  the  potential,  o  is  the  conductivity  tensor,  Iv  is  the 
volume  current  source  density.  Note  that  the  conductivity  is,  in  gen¬ 
eral,  a  tensor  quantity,  which  may  reflect  different  conductivities 
in  different  directions  (anisotropy),  and  a  function  of  position,  We 
will  consider  only  isotropic  models,  however ,  in  regions  free  of 
sources,  and  this  reduces  to  Laplaces'  equation, 
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Table  7 


1 ntere 1 ect rode 

Co r  r e 1  a t ion 

(  1  )  regressed 

vs  point  spread 

f  unc  t i on  . 

SPATIAL 

S  1  ope 

1 ntercept 

F-  rat i o  ( 2 ) 

OPERATOR 

Common  Average 

0 . 8878 

-0.2644 

1278.42 

Lap  1 ac i an 

0 .3742 

-0.  1200 

251.83 

Decon vo 1 u  t i on 

0 .3055 

-0  .  1002 

184.61 

(1)  Interelectrode  correlations  calculated  over  s t i mu  I  us  -  I ocked 
average  from  0.5  sec  before  to  1.0  sec  after  stimulus. 

(2)  Ratio  of  regression  to  residual  variance*  df  =  (1,1223). 
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Unfortunately,  a  knowledge  of  the  potentials  and  fields  at  the  scalp 
does  not  suffice  to  determine  the  sources;  a  given  distribution  of 
potential  (field)  at  the  scalp  (or  any  other  surfacet  including  the 
sources)  may  be  produced  by  infinitely  many  different  distributions 
of  sources.  In  order  to  resolve  this  ambiguity  some  additional  infor¬ 
mation  as  to  the  character!  orientation,  and/or  number  of  sources 
must  be  added  to  determine  the  sources  of  a  given  potential  (field) 
distribution.  One  convenient  approach  which  has  been  successfully 
applied  both  to  potentials  and  fields  is  to  model  the  sources  as  one 
or  a  few  equivalent  current  dipoles  whose  positions  and  orientations 
are  uncons t r a i ned  in  a  concent  r i c-sphere  model  of  the  tissue.  This 
has  been  applied  to  averages  of  sensory  ERPs  in  some  situations 
(Brazier,  1959;  Geisler  k  Gerstein,  1961;  Hosek,  1970;  Kavanagh, 
1972;  Ary,  1976;  Kavanagh  et  al.,  1978;  Oarcey  et  al . ,  1980)  and  to 
averages  of  ERFs  (reviewed  in  Williamson  k  Kaufman,  1981a, b;  Erne  et 
al.,  1981;  Williamson  et  al.,  1983;  Okada,  1983;  Weinberg  et  al., 
1985;  Willi amson  k  Kau  fman ,  In  press).  However ,  wh  Me  one  or  two 
equivalent  current  dipoles  may  be  adequate  to  model  the  major  source 
of  a  sensory  ERP  peak,  such  models  do  not  seem  realistic  for  even 
simple  goa I -d i rected  tasks.  Furthermore,  the  non - spher i ca I  topology 
of  the  actual  anatomy  may  result  in  considerable  distortion  of  the 
r  esu I t  s . 

More  realistic  head  and  brain  models,  as  well  as  additional  informa¬ 
tion,  are  required  to  attempt  multi-source  localization.  We  propose  a 
number  of  steps  to  achieve  these  requirements:  (1)  use  of  MR  I  scans 
and  f i n i t e - d i f f er ence  models  to  improve  the  head  and  brain  model;  (2) 
use  spatial  deconvolut ioon  to  reduce  spatial  smearing  of  potential 
measures;  (3)  use  of  the  MEG  to  provide  complementary  information; 
(4)  use  of  other  signal  processing  procedures  to  enhance  the  signal  - 
to-noise  ratio  of  both  EEG  and  MEG  (such  as  trial  selection  and  fil¬ 
tering);  (5)  use  of  time  dependencies  as  determined  from  the  event- 
related  covariances  of  fields  and  deconvolved  EEGs  to  constrain  the 
errors  in  source  modeling;  (6)  use  of  several  algorithmic  approaches, 
including  an  exhaustive  search  for  a  subset  of  physical  sources  from 
a  larger  candidate  set;  and  (7)  use  of  PET  blood  flow  data  (Budinger 
et  al.,  1984)  obtained  during  repetitions  of  the  same  task,  or  a  com¬ 
parable  one,  from  the  same  subjects  to  validate  the  results. 


6.  MEG  Data  Ana  I  vs i 5 .  A  co I  I abo r a t i on  is  underway  with 
the  laboratories  of  Drs.  Williamson  and  Kaufman,  and  Beatty.  The  pur¬ 
pose  of  this  effort  is  to  make  a  gateway  for  their  MEG  data  into  the 
ADIEEG  system.  We  are  developing  special  noise  suppression  digital 
filters,  and  will  apply  our  feature  extraction  and  pattern  recog¬ 
nition  analyses  to  several  data  sets. 
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